Effects of Model Complexity on Generalization Performance of Convolutional Neural Net- works

Convolutional neural networks are known to be effective in learning complex image classification tasks. However, how to design the architecture or complexity of the network structure requires a more quantitative analysis of the architecture design. In this paper, we study the effect of model complexity on generalization capability of the convolutional neural networks on large-scale, real-life digit recognition data. We used the digit images of the MNIST dataset to train the neural networks and evaluated their performance on a test set of unobserved images. Using the LeNet software tool we varied the number of hidden layers and the number of units in the layers to evaluate the effect of model complexity on the generalization capability of the convolutional neural networks. In our experimental settings, we observe robust generalization performances of the convolutional neural networks on a wide range of model complexities. We analyze and discuss how the convolution layer and the subsampling layer may contribute to the generalization performance.